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(54) Method and apparatus for evaluating a scale factor and a rotation angle in image processing 



(57) The present invention concerns a method for 
evaluating a scale factor and a rotation angle in image 
processing and more particularly digital image process- 
ing. The method includes the steps of first providing an 
input image, an then reducing the input image to provide 
a smaller image. The smaller image is scaled according 
to a plurality of factors to produce a plurality of scaled 
images and is also rotated according to a plurality of fac- 
tors to produce a plurality of rotated images. The scaled 
images in the rotated images are then placed into a mo- 
saic which is arranged by scale and rotation. A refer- 



enced template is provided and the referenced template 
is correlated with the mosaic of scaled and rotated im- 
ages to provide a scale factor and a rotation angle based 
on the position in the mosaic of the high score relation 
value between the reference template and the scaled 
and rotated images. The method is more efficient in that 
it reduces the amount of computations required to obtain 
useful results. The method also does not require object 
centering prior to performing the analysis and does not 
require complex mathematical expansion of the object 
under analysis. 
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Description 

Field of the invention 

[0001 ] The present invention relates to a method and 5 
apparatus for evaluating a scale factor and a rotation 
angle in image processing. 

Description of the prior art 

10 

[0002] In many optically automated applications, 
there is a need for the evaluation of the rotation angle 
or scale of an object. For example, if an electronic chip 
has to be soldered on a printed circuit board, the exact 
orientation of the object has to be known to ensure a *5 
good fit between the pins of the chip and the connectors 
of the printed-circuit board. 

[0003] In other applications, it is required to evaluate 
the distance of an object from a camera, such as for a 
robot to go pick up an object or, within a pattern recog- 20 
nition system, to compare the object with reference tem- 
plates at the correct scale factor. The distance between 
the object and the camera is directly proportional to the 
scale of the object observed by the camera. Knowing 
the scale of an object compared to a reference is equiv- 25 
alent to knowing the distance. In both cases this infor- 
mation is useful for further processing, such as in the 
case of pattern recognition, target tracking and other ap- 
plications. 

[0004] A fair amount of research has been done in this 30 
field. Among others, US patent no. 5,067,019 to Juday 
et al. entitled "Programmable remapper for image 
processing" describes an image processing system for 
realising image preprocessing for maculopathy. It is also 
suggested that the pre-processor described in this pat- 35 
ent could also be useful for optical correlators. However, 
the different functions described are the usual log-polar, 
ring-wedge, log-log mathematical operations. The prob- 
lem with these functions is that they give rise to the prob- 
lem of image centering. Because these techniques give 40 
different object representations for different centers of 
expansion, the object has to be centered conveniently 
before the recognition can be property effected. 
[0005] Most of the other known techniques are de- 
rived from the field of pattern recognition where an ob- *s 
ject has to be recognised with invariance to scale, rota- 
tion and size. One of these techniques is based on the 
expansion of the object in mathematical components 
(see, for example, Y.N. Hsu and H. H. Arsenault, "Pat- 
tern Discrimination by Multiple Circular Harmonic Com- so 
ponents", Appl. Opt., 23, 841 , (1984)). Othertechniques 
are based on geometrical or mathematical transforma- 
tion of an object, such as the techniques described in Y 
Sheng, C. Lejeune, H. H. Arsenault, "Frequency-do- 
main Fourier-Mellin descriptors for invariant pattern rec- 55 
ognition", Optical Engineering, 27, 5, 354, (1988), and 
D. Casasent, S-F X., A. J. Lee, J.-Z Song, "Real-time 
deformation invariant optical pattern recognition using 



2 

coordinate transformations", Applied Optics, 26, 5, 
1987. 

[0006] Invariant moments are also used for these 
kinds of operations, and in this case, the moments of 
inertia are computed according to a center of expansion . 
This technique is described in Q. Zhan, T. Minemoto, 
"Moment invariants for pattern recognition under trans- 
lation, rotation, and changes of size and brightness", 
Journal of Modern optics, 40, 11, 2267, 1993. 
[0007] These methods suffer from different problems 
that can be itemized as follows: 

[0008] The gain of an invariance, for instance rotation, 
will be accompanied by a loss of the robustness for an- 
other invariance, for example scale robustness; 
[0009] The gain of an invariance is often accompa- 
nied by a loss of information on the object; 
[0010] The information on the rotation angle or scale 
factor is lost with the adjunction of invariance, so one is 
able to locate an object but is not easily able to identify 
its rotation angle or scale factor; 

[0011] The evaluation of the angle of rotation or scale 
factor thus requires display of multiple filters, increasing 
the processing time. As such, the time required to eval- 
uate 1024 rotation angle or scale factors is 1024 times 
greater than the one required for one rotation angle and 
one scale factor. 

[0012] Most of these methods rely on finding an ap- 
propriate center of expansion of an object, which itself 
requires considerable computations; and 
[0013] When a technique requires a center of expan- 
sion, it is then intrinsically not able to operate in the pres- 
ence of multiple objects in the input plane. 

Summary of the invention 

[0014] It is an object of the invention to provide a 
method and apparatus for performing scale factor and 
rotation evaluation in an image processing system, 
which reduces the amount of computations required to 
obtain useful results. Also, the invention is concerned 
with a method and apparatus which does not require ob- 
ject centering priorto performing the analysis, and which 
does not require complex mathematical expansion of 
the object under analysis. Other objects will become ap- 
parent hereinafter. 

[0015] In accordance with the Invention, this object is 
achieved with: 

a method for evaluating a scale factor and a rotation 
angle in image processing, comprising the steps of: 

(a) providing an input image; 

(b) reducing the input image to provide a small- 
er image; 

(c) scaling the smaller image according to a plu- 
rality of factors to produce a plurality of scaled 
images; 

(d) rotating the smaller image according to a 
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plurality of factors to produce a plurality of ro- 
tated images; 

(e) placing the scaled images and the rotated 
images into a mosaic arranged by scale and ro- 
tation; 

(f) providing a reference template; 

(g) correlating said reference template with said 
mosaic of scaled and rotated images; and 

(h) providing a scale factor and a rotation angle 
based on the position in the mosaic of the high- 
est correlation value between the reference 
template and the scaled and rotated images; 

a method for evaluating a scale factor and a rotation 
angle in image processing, comprising the steps of: 

(a) providing an input image; 

(b) reducing the input image to provide a small- 
er image; 

(c) providing a reference template; 

(d) scaling the reference template according to 
a plurality of factors to produce a plurality of 
scaled images; 

(e) rotating the reference template according to 
a plurality of factors to produce a plurality of ro- 
tated images; 

(f) placing the scaled images and the rotated 
images into a mosaic arranged by scale and ro- 
tation; 

(g) correlating said smaller image with said mo- 
saic of scaled and rotated images; and 

(h) providing a scale factor and a rotation angle 
based on the position in the mosaic of the high- 
est correlation value between the smaller im- 
age and the scaled and rotated images. 

[0016] Another aspect of the invention concerns a 
system for evaluating a scale factor and a rotation angle 
in image processing, comprising: 

(a) input means for receiving an input image; 

(b) means for reducing the input image to provide a 
smaller image; 

(c) means for scaling the smaller image according 
to a plurality of factors to produce a plurality of 
scaled images; 

(d) means for rotating the smaller image according 
to a plurality of factors to produce a plurality of ro- 
tated images; 

(e) means for receiving and placing the scaled im- 
ages and the rotated images into a mosaic arranged 
by scale and rotation; 

(f) means for providing at least one reference tem- 
plate; 

(g) means for correlating each of said at least one 
reference template with said mosaic of scaled and 
rotated images; and 

(h) means for providing a scale factor and a rotation 



angle based on the position in the mosaic of the 
highest correlation value between each of the ref- 
erence templates and the scaled and rotated imag- 
es; 

[0017] Alternatively, the present invention concerns a 
system for evaluating a scale factor and a rotation angle 
in image processing, comprising: 

(a) input means for receiving an input image; 

(b) means for reducing the input image to provide a 
smaller image; 

(c) means for providing at least one reference tem- 
plate; 

(d) means for scaling each of the at least one refer- 
ence template according to a plurality of factors to 
produce a plurality of scaled images; 

(e) means for rotating each of the at least one ref- 
erence template according to a plurality of factors 
to produce a plurality of rotated images; 

(f) means for placing the scaled images and the ro- 
tated images into a mosaic arranged by scale and 
rotation for each of the reference template; 

(g) means for correlating said smaller image with 
each of said mosaic of scaled and rotated images; 
and 

(h) means for providing a scale facto rand a rotation 
angle based on the position in the mosaic of the 
highest correlation value between the smaller im- 
age and the scaled and rotated images; 

[0018] Finally, a method for evaluating a scale factor 
and a rotation angle in image processing is described, 
comprising the steps of: 

(a) providing an input image; 

(b) scaling the image according to a plurality of fac- 
tors to produce a plurality of scaled images; 

(c) rotating the image according to a plurality of fac- 
tors to produce a plurality of rotated images; 

(d) placing the scaled images and the rotated imag- 
es into a mosaic arranged by scale and rotation; 

(e) providing a reference template; 

(f) correlating said reference template with said mo- 
saic of scaled and rotated images; and 

(g) providing a scale factor and a rotation angle 
based on the position in the mosaic of the highest 
correlation value between the reference template 
and the scaled and rotated images. 

Brief description of the drawings 

[0019] The present invention and its advantages will 
be more easily understood after reading the following 
no n- restrictive description of preferred embodiments 
thereof , made with reference to the following drawings 
in which: 
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Figures 1a, 1b and 1c are schematic representa- 
tions of the steps of the method according to a pre- 
ferred embodiment of the invention, of a pictorial 
representation of the application of the method for 
an unsealed and not rotated object, and of a pictorial 
representation of the application of the method for 
a scaled and rotated object; and 
Figures 2a and 2b are schematic representations 
of the steps of the method according to a variation 
thereof, and of a pictorial representation of the ap- 
plication of the method according to this variation. 

Description of a preferred embodiment of the 
invention 

[0020] The present invention finds application in the 
field of image processing, and more precisely in pattern 
recognition and target tracking. The invention can be 
useful in applications requiring rotation angle evalua- 
tion, scale evaluation, or distance evaluation. It can also 
be useful in pattern recognition applications implying a 
change of rotation, scale or distance of the object tar- 
geted (or of the camera used to observe the scene). 
[0021] The invention is a processing technique forthe 
evaluation of the angle of rotation of an object, the scale 
of an object or the distance of an object. Given an image 
of an object, it will give the rotation angle, and the scale 
or the distance of the object according to a reference 
template. This information can then be further used to 
remap and recognize an object. 

[0022] Another field in which the invention may be 
useful is in the evaluation of the out of plane rotation of 
a three dimensional object observed with a camera. It 
is often required to know the angle of rotation of the ob- 
ject both according to the plane perpendicular to the 
camera and to the plane parallel to the camera. 
[0023] The technique developed forthe rotation angle 
(in-plane and out of plane), and the scale evaluation (ei- 
ther isomorphic or anamorphic) and/or the distance 
evaluation (which is equivalent to scale evaluation) will 
be referred herein as "RS evaluation" technique. The 
RS technique is achieved using reduction of the resolu- 
tion of the image grabbed : followed with a mapping of 
the information to different scales and rotations and 
completed finally by a correlation with a reference tem- 
plate. 

[0024] The method includes the following basic steps. 
First, an unknown input image is reduced in resolution 
to provide a smaller image. The smaller the image, the 
larger the number of scale factor and rotation angle that 
can be evaluated. If the object Is already small enough 
to accommodate multiple replica in the same image then 
the reduction step can be eliminated. For example, the 
resolution reduction can be either performed by sam- 
pling, mean, or other algorithms. The smaller image is 
then scaled and rotated according to different values. 
This step can easily be performed with the help of a look- 
up-table with corresponding positions. Look-up-table 



mapping can be performed with simple memory ad- 
dressing operations and is not a time-consuming oper- 
ation. 

[0025] Using standard co-ordinate transformation inv 

5 plies multiple computations. However, the relationship 
between a rotated and scaled image can be pre-com- 
puted. When computing the scale and rotation change, 
if the correspondence between the original image pixels 
position and the rotate d-sca led image pixels position is 

10 pre-computed, then the rotation-scale change can be 
applied simply by changing position of the original image 
pixels. It should be understood that the look-up table is 
applied to the position of the pixels rather than to the 
value of the pixels. 

is [0026] The look-up-table can also accommodate for 
out-of-plane rotations. For instance, if an object with al- 
most planar front surface (the surface directed toward 
the camera) is rotated around the axis parallel to its sur- 
face, then the change of aspect perceived by the cam- 

20 era can be expressed by simple anamorphic scale 
change, i.e. the object will have the same height, but will 
appear thinner. Moreover, if the surface is mostly planar 
or the general three dimensional model of the object is 
available, an out-of-plane rotation can be approximated 

25 by simple look-up-table corresponding to position 
changes. 

[0027] A large image (mosaic) containing all the rep- 
licas of the input images (rotated and scaled) is then 
generated. The range of angle and scale evaluation de- 

30 pends on the number of small images than can be tiled 
in the mosaic. This is dependant on both the size of the 
small replicas and the size of the image containing the 
whole mosaic. Once the mosaic is generated, the whole 
image is compared by correlation means to a reference 

35 template. 

[0028] The reference template is used to generate a 
filter via a Fourier transform operation. The reference 
template can be pre-computed. Usually the small replica 
of the reference with a 1 .0 scale factor and no rotation 
is used to perform the correlation. The correlation can 
be either performed digitally or optically, using known 
means. 

[0029] The correlation plane will provide a mosaic of 
correlations. The correlation plane is divided into sub- 
sections each corresponding to a scale factor and a ro- 
tation angle. By evaluating the maximum correlation val- 
ue, an evaluation of the scale factor and the rotation an- 
gle (according to the reference template) is performed. 
It is to be noted that if the image is scaled by a factor of 
90% the stronger correlation will be observed at 
1 .0/0.90. Similarly, if the image is rotated by 10 degrees, 
the maximum correlation will appear at -10 degrees. 
[0030] Referring now to Fig. 1 a, there is shown the 
steps of the method. Figure 1b pictorially shows the ap- 
plication of the method. The input image 11 is reduced 
to produce a reduced image 13. The reduced image is 
rotated and scaled according to different factors and ar- 
ranged in a mosaic 1 5. In this case, the top row of the 
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mosaic represents the reduced image scaled to detect 
1 .1 times and rotated to detect 10, 0 and -1 0 degrees, 
respectively. The middle row and the bottom row are 
equivalent, but for scale factors of 1.0 and 0.9 of the 
reduced image. A reference template 1 7, is used to gen- 5 
erate a filter through a Fourier Transform operation. The 
reference template 17 is correlated with the mosaic 15. 
This operation produces a mosaic of correlations, and 
by identifying the highest correlation, information about 
the rotation angle and scale of the object is provided. 10 
Fig. 1b produces a result where the rotation angle is 0 
degrees, and the scale factor 1 .0. However, Fig. 1c pro- 
duces a result where the rotation angle is 10 degrees, 
and the scale factor 0.9. 

[0031] A slight modification of the technique, illustrat- 15 
ed in Fig. 2, can also be used to roughly evaluate the 
parameters mentioned above. In this case, the object 
acquired (input image) 101 is reduced in dimensions 
103 and a filter is generated from the Fourier transform 
of the object 105. Because the object is reduced in di- 20 
mension, the Fourier transform will be less time con- 
suming than the Fourier transform on larger images. 
[0032] A mosaic of images in the database 1 07, cre- 
ated either from the different scales of an object, the dif- 
ferent rotations or the different aspects (for instance 25 
from different three dimension points of view) is present- 
ed as the input image. From the one step correlation, 
the most corresponding pattern can be identified 109, 
and consequently the rotation angle, the scale, the types 
or the aspect of the object are known. Further process- 30 
ing can thus be realized to confirm or refine the answer. 
[0033] Once the rotation, the scale, or the aspect of 
the object have been evaluated by any of the two pre- 
vious techniques, the original object can be rescaled, 
rotated or remapped and then compared at high reso- 35 
lution with correlation means for final identification. 
[0034] The present invention thus concerns a rotation 
or scale evaluation technique comprising, as main 
steps, the reduction of the resolution of a scene, the cre- 
ation of a mosaic of small images rotated, scaled, or pro- 40 
jected; the correlation of the images with a reduced pat- 
tern; and the identification of the rotation or scale ac- 
cording to the results. The rotation of the objects can be 
in-plane or out-of-plane, and the scale change can be 
either isomorphic or anamorphic. The mosaic can in- 45 
elude objects of different aspects and where the images 
of the objects can come from different points of view. 
[0035] The method according to the present invention 
can be either performed numerically or optically, in its 
entirety or partially. so 
[0036] The RS technique according to the present in- 
vention is based on the creation of a mosaic in the input 
plane, avoiding the reduction of diffraction efficiency and 
allowing the use of fast and simple look-up-table oper- 
ation for RS evaluation. There is thus no need for log- 55 
polar or log-log conversions to achieve rotation and 
scale invariance, nor any need for object centering. As 
such, the translation invariance of the correlation oper- 



ation is not lost due to change of the object aspect with 
a change of the center of representation. 
[00371 The overall technique is rotation, scale and 
translation invariant, and the information on scale factor, 
rotation angle and position is conserved. 
[0038] The use of a single correlation gives the eval- 
uation of the rotation angles and scale factors over a 
wide range reducing the overall processing time. 
[0039] There is no needfor complex mathematical ex- 
pansion of the objects and multiple correlations are per- 
formed within a single frame. 

[0040] The resolution reduction should be high 
enough for the evaluation of multiple scale factor or ro- 
tation angle. However it should be small enough to pre- 
clude loss of critical information of the object. For in- 
stance, an object of 256x256 pixels with a 2 times re- 
duction factor gives evaluation of 4 parameters with 
small images of 128x128 pixels. Given a 32 times re- 
duction factor, a mosaic of 32x32 objects, each showing 
8x8 pixels, can be generated, giving an evaluation of 
1024 parameters (scale, rotation or else). However, a 
reduction factor of 64 would reduce the smaller images 
to 4x4 which would not be sufficient enough for most 
parameter evaluations. The resolution factor as such 
should be chosen to conserve enough information to 
evaluate the required parameter. 
[0041] The correlation can either be performed on nu- 
merical platform or optical correlator. 
[0042] It should be understood that the method and 
system of the present invention can also be easily mod- 
ified to accommodate any number of reference tem- 
plates, for example in the case where the image 
processing system is adapted to perform recognition of 
various objects, by repeating the steps of the method 
for each different reference template. Recognition will 
be presumed with a predetermined level of certainty 
when the correlation value is above a threshold level. 
[0043] Finally, the method and system of the present 
invention are preferably embodied in a computer pro- 
gram. 

[0044] Although the present invention has been ex- 
plained hereinabove by way of a preferred embodiment 
thereof it should be pointed out that any modifications 
to this preferred embodiment within the scope of the ap- 
pended claims is not deemed to alter or change the na- 
ture and scope of the present invention. 



Claims 

1 . A method for evaluating a scale factor and a rotation 
angle in image processing, comprising the steps of: 

(a) providing an input image; 

(b) reducing the input image to provide a small- 
er image; 

(c) scaling the smaller image according to a plu- 
rality of factors to produce a plurality of scaled 
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images: 

(d) rotating the smaller image according to a 
plurality of factors to produce a plurality of ro- 
tated images; 

(e) placing the scaled images and the rotated 
images into a mosaic arranged by scale and ro- 
tation; 

(f) providing a reference template; 

(g) correlating said reference template with said 
mosaic of scaled and rotated images; and 

(h) providing a scale factor and a rotation angle 
based on the position in the mosaic of the high- 
est correlation value between the reference 
template and the scaled and rotated images. 

2. A method according to claim 1 , wherein said meth- 
od further includes step (f.1) performing a Fourier 
transform on said reference template. 

3. A method according to claim 1 , wherein said step 
(c) of scaling the reduced images comprises an iso- 
morphic scale change. 

4. A method according to claim 1 , wherein said step 
(c) of scaling the reduced image comprises an an- 
amorphic scale change. 

5. A method according to claim 1 , wherein said rota- 
tion is an in -plane rotation. 

6. A method according to claim 1 , wherein said rota- 
tion is an out-of -plane rotation. 

7. A method according to claim 1 , further comprising 
the steps of: 

(i) remapping the input image to its input size; 
and 

(j) comparing the remapped image to a refer- 
ence template at a higher resolution for recog- 
nising the input image. 

8. A method according to claim 1 , wherein said step 
(j) includes correlating the remapped image to a ref- 
erence template. 

9. A method for evaluating a scale factor and a rotation 
angle in image processing, comprising the steps of: 

(a) providing an input image; 

(b) reducing the input image to provide a small- 
er image; 

(c) providing a reference template; 

(d) scaling the reference template according to 
a plurality of factors to produce a plurality of 
scaled images; 

(e) rotating the reference template according to 
a plurality of factors to produce a plurality of ro- 



tated images; 

(f) placing the scaled images and the rotated 
images into a mosaic arranged by scale and ro- 
tation; 

5 (g) correlating said smaller image with said mo- 

saic of scaled and rotated images; and 

(h) providing a scale factor and a rotation angle 
based on the position in the mosaic of the high- 
est correlation value between the smaller im- 

10 age and the scaled and rotated images. 

10. A method according to claim 9, wherein said meth- 
od further includes step (b.1) performing a Fourier 
transform on said smaller image. 

15 

11. A method according to claim 9, wherein said step 
(d) of scaling the reference template comprises an 
isomorphic scale change. 

20 12. A method according to claim 9, wherein said step 
(d) of scaling the reference template comprises an 
anamorphic scale change. 

13. A method according to claim 9, wherein said rota- 
25 tion is an in-plane rotation. 

14. A method according to claim 9, wherein said rota- 
tion is an out-of-plane rotation. 

30 15. A method according to claim 9, further comprising 
the steps of: 

(i) remapping the input image to its input size; 
and 

35 (j) comparing the remapped image to a refer- 

ence template at a higher resolution for recog- 
nising the input image. 

16. A method according to claim 1, wherein said step 
40 (j) includes correlating the remapped image to a ref- 
erence template. 

17. A system for evaluating a scale factor and a rotation 
angle in image processing, comprising: 

45 

(a) input means for receiving an input image; 

(b) means for reducing the input image to pro- 
vide a smaller image; 

(c) means for scaling the smaller image accord- 
so ing to a plurality of factors to produce a plurality 

of scaled images; 

(d) means for rotating the smaller image ac- 
cording to a plurality of factors to produce a plu- 
rality of rotated images: 

55 (e) means for receiving and placing the scaled 

images and the rotated images into a mosaic 
arranged by scale and rotation; 
(f) means for providing at least one reference 



20 12. 

13. 

25 

14. 
30 15. 

35 

16. 



BNSDCCID: <EP 1152371A2J_> 



11 BP 1152 371 A2 12 



w 



template; 

(g) means for correlating each of said at least 
one reference template with said mosaic of 
scaled and rotated images; and 

(h) means for providing a scale factor and a ro- 
tation angle based on the position in the mosaic 
of the highest correlation value between each 
of the reference templates and the scaled and 
rotated images. 

18. A system for evaluating a scale factor and a rotation 
angle in image processing, comprising: 

(a) input means for receiving an input image; 

(b) means for reducing the input image to pro- 15 
vide a smaller image; 

(c) means for providing at least one reference 
template; 

(d) means for scaling each of the at least one 
reference template according to a plurality of 20 
factors to produce a plurality of scaled images; 

(e) means for rotating each of the at least one 
reference template according to a plurality of 
factors to produce a plurality of rotated images; 

(f) means for placing the scaled images and the 25 
rotated images into a mosaic arranged by scale 
and rotation for each of the reference template; 

(g) means for correlating said smaller image 
with each of said mosaic of scaled and rotated 
images; and 30 

(h) means for providing a scale factor and a ro- 
tation angle based on the position in the mosaic 
of the highest correlation value between the 
smaller image and the scaled and rotated im- 

35 



19. A method for evaluating a scale factor and a rotation 
angle in image processing, comprising the steps of; 

(a) providing an input image; 40 

(b) scaling the image according to a plurality of 
factors to produce a plurality of scaled images; 

(c) rotating the image according to a plurality of 
factors to produce a plurality of rotated images; 

(d) placing the scaled images and the rotated *s 
Images into a mosaic arranged by scale and ro- 
tation; 

(e) providing a reference template; 

(f) correlating said reference template with said 
mosaic of scaled and rotated images; and so 

(g) providing a scale factor and a rotation angle 
based on the position in the mosaic of the high- 
est correlation value between the reference 
template and the scaled and rotated images. 
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